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Abstract 

In  this  work  we  consider  the  discrete  time  detection  of  strong  mixing  signals  in 
strong  mixing  noise,  and  we  allow  a  large  degree  of  dependency  to  exist  between 
the  signal  and  the  noise.  We  investigate  the  memoryless  detector  which  is 
optimum  in  the  sense  of  the  asymptotic  relative  efficiency.  It  is  shown  that 
the  design  of  this  detector  reduces  to  the  solution  of  an  integral  equation  in 
which  knowledge  of  only  the  second-order  statistics  of  the  random  processes 
involved  is  required. 


I.  INTRODUCTION 

The  detection  of  signals  in  corrupting  noise  has 
been  an  area  of  interest  for  some  time.  Because 
of  modern  high  speed  sampling,  it  is  expected  that 
the  underlying  random  processes  involved  will  not 
be  "white",  but  will  instead  possess  dependency  to 
a  certain  degree.  Neyman-Pearson  optimal 
techniques  [ 1 ]  are  tractable  only  in  cases  where 
the  appropriate  multivariate  distributions  are 
known.  In  many  non-Gaussian  situations  these 
distributions  are  not  known,  which  has  thus  led  to 
the  choice  of  an  alternate  fidelity  criterion, 
commonly  the  asymptotic  relative  efficiency  (ARE) 
criterion,  which  is  especially  appropriate  in  the 
weak  signal  and  large  sample  situation.  Because 
continuous  time  detection  is  often  intractable  in 
the  non-Gaussian  case,  current  efforts  are 
directed  toward  discrete  time  detection.  Results 
in  this  area  have  been  obtained  recently  by  Poor 
and  Thomas  [2,31  for  the  case  of  memoryless 
detection  of  a  known  constant  signal  in  additive 
m-dependent  noise;  we  have  shown  [4,5]  how  these 
results  may  be  extended  to  a  large  class  of 
^-mixing  noises.  Because  an  assumption  of  a 
constant  signal  is  in  many  cases  overly 
restrictive,  we  have  also  considered  [6]  the  case 
where  both  the  signal  and  noise  may  be  modeled  as 
4)-mixing  processes,  where  a  large  degree  of 
dependency  may  also  occur  between  the  signal  and 
noise.  All  of  these  results  have  the  advantage  of 
requiring  only  second-order  statistical  knowledge 
of  the  random  processes  involved. 

The  employment  of  the  <J>-mixing  models  of  [4-6]  is 
motivated  by  an  expectation  that  dependency 
between  samples  gradually  "decreases"  as  the 
samples  are  more  widely  separated  in  time,  and  the 
formal  definition  of  a  ^-mixing  process  is 
consistent  with  such  a  property.  The  class  of 
4>-mixing  processes  employed  may  be  seen  to  be 
quite  general,  however  as  described  in  the  next 
section,  the  formal  definition  is  more  restrictive 


than  what  we  might  expect  as  the  most  natural 
consequence  of  our  intuition.  In  this  paper  we 
will  model  the  signal  and  noise  in  a  way  which  is 
in  many  ways  much  more  consistent  with  our 
intuition.  This  will  be  achieved  through  the 
employment  of  strong  mixing  processes  to  model  the 
signal  and  noise.  The  class  of  strong  mixing 
processes  is  more  general  than  that  of  ^-mixing 
processes,  and  because  of  ties  to  the  maximal 
correlation  coefficient  the  validity  of  a  strong 
mixing  model  is  easier  to  check.  We  therefore 
will  consider  the  general  situation  where  we  are 
detecting  the  presence  of  a  strong  mixing  signal 
in  strong  mixing  noise. 

II.  PRELIMINARIES 

Let  (X..;  1=1,2,...}  be  a  strictly  stationary 

sequence  of  random  variables.  For  a£b,  define 
M(a,b)  =  o{Xa *xa+l  *  *  *  *  ,Xb> *  t^ie  a~al9el:)ra 

generated  by  the  indicated  random  variables,  where 
a  and  b  may  take  on  extended  real  values.  Then 
{ X.. ;  1=1,2,..,}  is  symmetrically  4>-mixing  if  there 

exists  a  nonnegative  sequence  {4.;  i-1,2,...}  with 
4>.  +0  such  that  for  each  k,  l£k<°°,  and  for  each 
1>,1,  Ej«M(l,k)  and  E2eM(k+i,°°)  together  imply 

|P(E1nE2)-P(E1)P(E2)|  <  ♦1  min  {P(Ej) ,P(E?) } . 

In  [4-6]  the  above  type  of  process  is  employed. 

Note  that  the  left  side  of  the  above  inequality 
provides  a  measure  of  dependence  between  events  E, 
and  E£,  and  the  right  side  bounds  this  quantity  1 

with  a  term  involving  Ej  and  E2.  Such  a  definition 

has  computational  advantages;  for  example,  it 
results  in  the  very  powerful  Lemma  1  of  [7 .  p. 170] - 
However,  it  is  a  stronger  requirement  than  our 
intuition  might  demand.  Since  we  really  wish  to 
simply  require  a  "decrease"  in  dependency  as  Ej 
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and  E2  are  more  widely  separated  in  time,  it  is 

thus  more  natural  to  employ  the  weaker  requirement 
that  there  exists  a  nonnegative  sequence 
{a.;  i=l,2#...}  with  a..  +  0  such  that  for  all 

and  E2  as  above  we  have 

|P(EinE2)-P(E1)P(E2)|  <  a.. 

A  process  satisfying  this  condition  is  called 
strong  mixing.  We  will  consider  the  detection  of 
a  strong  mixing  signal  {S.;  i=l,2,...},  where 
2  1 

0  <  E{Sj}  <  »,  in  additive  strong  mixing  noise 
{N.j ;  1=1,2,...},  where  we  observe  realizations 
{y.| ;  1  =  1,2,. ,.,n}  of  the  process  {Y. si=l,2,...,n}. 

In  order  to  apply  the  ARE  fidelity  criterion,  this 
will  amount  to  a  choice  between  the  two  hypotheses 

H0:  Yi  =Ni;  1  =  1»2’  —  >n 
Hl:  Yi  =Ni  +0Si;  1=1’2*-*-*n 


prime  concern. 

We  will  restrict  the  class  *8  of  nonlinearities  g 
to  include  those  measurable  real  valued  functions 
for  which  we  can  find  >  0  and  <5j>6  such  that 

the  random  variable  g(Nj+eSj)  satisfies 

E{ IgtNj+eSj)  |2+<5l>  <  «>  for  all  6  «  [0,6^,  and  such 

that  the  following  mild  regularity  conditions  hold, 
where  EQ(*)  denotes  expectation  computed  under  Hj 

with  parameter  0  (by  proper  choice  of  the 
threshold,  we  assume  without  loss  of  generality 
that  the  random  variables  g ( N ^ )  are  zero  mean): 

(^)  jg(x)f'(x)dx  /  0 


if  the  signal  and  noise  processes  are  independent; 


(b) 


1t.  [j  WY)}|eJ  a 
n—  nEQ{[Tg(Y)]2} 


n(g)  >  0 


where  0  is  a  parameter  which  will  be  allowed  to 
approach  zero  at  the  proper  rate,  thus  yielding 
the  asymptotic  limit.  Throughout  the  discussion 
we  will  assume  that  both  the  noise  and  signal 
processes  possess  (possibly  different) 
a-representations  which  satisfy 

l  a.  6/(2+6)  <  » 

1-1  1 

for  some  appropriate  6>  0.  Such  a  strong  mixing 
process  will  be  called  6-acceptable.  For 
convenience  we  assume  the  existence  of  densities 
fj ( • .  • )  Of  Nk  and  Nk+j,  f(-)  of  Nj,  f(*,*)  of  Nk 

and  S^,  where  the  latter  is  assumed  to  be 
independent  of  k.  We  also  assume 

K'x.y)  =  l  Lf^x.y)  +  My,x)]//f(x)f(y) 

j=l  J  J 

is  square  integrable  for  all  n,  and  that  f(-)  is 
strictly  positive  on  the  real  line.  We  assume  in 
addition  that 
2 

ly2  f(x,y)dy//f(x) 

3x 


and 

Jy  ~  f(x,y)dy//FTxT 

are  square  integrable.  Note  that  if  the  signal 
and  noise  are  independent,  the  latter  condition 
is  equivalent  to  the  assumption  of  finite  Fisher's 
information  number  contained  in  [2,3]  and  [4,5]. 
We  also  assume  that 

lim  j f( x-6y ,y)dy  =  f(x). 

9->0 

As  in  [2-6],  we  will  optimize  over  the  class  of 
optimal  memoryless  detectors  designed  under  a 
"white  noise"  assumption,  i.e.  where  a  test 
n 

statistic  T  (y)  =  7  g(y.)  is  compared  to  a 

9  1  =  i  1 

threshold.  Specifying  g  will  therefore  be  of 


if  JJy g(x)  j-  f(x,y)dxdy  ^  0,  or 


( b * )  lim  ^ 


MU*)) 


l2 

9=0J 


=  n(g)  >  0 


n—  nE0{[Tg(Y)]‘} 
if  JJyg(x)  ^  f(x,y)dxdy  =  0; 

(c)  ■  °°  <  lim  //R 

”  af  E{9(Ni+0S1)>le=o  "  ” 

for  some  constant  k.  >  0 
if  //yg(x)  J-  f(x,y)dxdy  f  0,  or 

,2 

(c 1 )  -»<  lim  -2-2  E{g(N1+0S1 ) } j  , 
n  +»  30  1  1  l6=k2/n,s 

=  ~2  E^g( Ni+0Si ) }  I e=o  <  00 
do 

for  some  constant  k2 > 0 
if  JJyg(x)  j;  f(x,y)dxdy  -  0; 

(d)  lim  EfgtN.+OS. )2+<S}  =  Etg(N, )2+<S>; 
0-0  1  1 

(e)  Jq  JJ g(x)f(x-0y,y)dxdy|0=o 

*  ff  Jq  g(x)f(x-6y,y)|0=o  dxdy 
if  JJyg(x)  ^  f(x,y)dxdy  t  0,  or 


X 

r 


s  on _ 


Dlst 


A 


ion/ 

Lity  Codes 
LI  and/or 
special 
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(e‘)  -^2  fj  g(x)f(x-0y,y)dxdy|Q_o 

30 

=  11^2  gi*)f(x-ey,y)ie=0  dxdy 
if  JJyg(x)  ^  f(x,y)dxdy  =  0; 


(f)  a2(g)  =  E{g(N,)2}  +  2  J  E  { g(  N, )  g(  N  i+, ) }  >  0. 

j=l  J 

Properties  (a)-(c')  are  assumptions  conventionally 
imposed  for  application  of  the  Pitman-Noether 
theorem  [8],  whereas  (d)-(e')  are  exceedingly  mild 
restrictions .  For  a  large  class  of  processes, 
including  all  of  the  examples  of  [3],  property  (f) 
is  satisfied  and  may  therefore  be  ignored.  Note 
that  these  assumptions  are  the  analog  to  those  of 
[6]. 

III.  DEVELOPMENT 

The  great  appeal  of  the  asymptotic  fidelity 
criterion  employed  is  that  it  allows  appeal  to 
central  limit  theorem  results.  We  first  need  the 
following  lemma,  which  is  the  analog  to  Lemma  1 
of  16] : 

Lemma  1:  If  {N. ;  1=1,2* _ }  and  {S.;  1=1,2, _ > 

are  6-acceptable  and  independent  processes,  then 
the  process  Nj  ’$1  >^2*^2' *  *  ’  1S  <5-accePtable. 

Proof:  We  may  repeat  the  first  part  of  the  proof 
of  Lemma  1  of  [6]  to  obtain 

|E {Ir  h  . }  -  E { I F  ] E { h  . } | 
tj  j  j 

I  I  f{/whjdFN(k,j)-JwdFN  JhjdFN(j)}dFs(k,j)| 

+  |  JJwdFN  fhjdFN(j)dFs(k,j) 

-  ////  *hjdFNdFN( J)dFsdFs( j) I . 

where  we  employ  the  notation  of  [6].  Application 
of  Lemma  1.2^of  [ 9 J  together  with  the  fact  that 
|w|£l  and  | h j  |  <_  1  a.s.  then  shows  that  first 

summand  on  the  right  side  of  the  inequality  can  be 
upper  bounded  by  4a.,  where  ;  1=1,2,...}  is  an 

m-representation  for  {N^ ; i  =  1 ,2 , . . In  a  similar 

manner  the  second  summand  can  be  upper  bounded  by 
46,j ,  where  { 6^ ;  i  =  l,2,...}  is  an  a- representation 

for  {S^;  1=1,2,...}.  We  therefore  obtain 


Lemma  2:  Suppose  (S,.;  1=1,2,..*}  is  6-acceptable, 
and  for  a  fixed  nonnegative  integer  m,  N..  =  G( X - ) 
for  1=1,2,...  where  X.  is  o{S.  m»...,S  } 

«  i  i-m  i+m 

measurable,  G:  1R  +  1R  is  measurable,  and 

(Z . ;  1=1,2,...}  is  6-acceptable  and  independent  of 

{S^ ;  1=1,2,...}  (we  let  S.  m  =  Sj  for  i  <_m) . 

Then  N^S^N^^^, . . .  is  6-acceptable. 

Proof:  This  follows  through  an  argument  identical 
to  the  proof  of  Lemma  2  of  [6].  Q.E.D. 

We  can  thus  obtain  a  useful  result: 

Theorem  1:  Suppose  (./. :  1=1 ,2, . . . }  is  a 

family  of  measurable  functions  where  p  is  a  fixed 
nonnegative  integer.  Then  under  the  hypothesis  of 
Lemma  1  or  Lemma  2,  we  have  that  (  ^(N..  ,S. , . . . , 

N.+p*S.+p);  1=1,2,...}  is  6-acceptable. 

Proof:  This  follows  as  a  consequence  of  Lemma  1, 
Lenina  2,  and  a  straightforward  modification  of 
Proposition  7  of  [5].  Q.E.D. 

We  can  now  obtain  the  result  which  will  allow 
employment  of  the  Pitman-Noether  Theorem  [8]. 

Theorem  2:  Suppose  0  «  ]R  with  9  -+  0,  and  q  «  (8. 
-  n  n 

Let  T  n  =  T  //n  under  H.  with  parameter  Q  ,  where 
n,9  g  1  n 

the  noise  and  signal  processes  satisfy  the 
hypothesis  of  Lemma  1  or  Lemma  2.  If 

<Ve“E{(Tn,9-E{Tn,e3>2>*  the" 


n,t 


E{Tn,e} 


n,( 


— > 


N( 0, 1) . 


Proof:  Letting  T  „  =  under  HA  it  follows 

-  *  n,0  g'  n  0  2 

from  condition  (f)  that  lim  a  A  =  of.(g)  >  0, 

n  +00  n,0  0vy/ 

2 

and  hence,  lim  n  °n  q  =  00 •  thus  obtain  ^rom 

n  -*■ 00  ’ 


Theorem  1.4  of  [9]  and  Theorem  1  that 

•fiT  n  </ 

— — “  — N(0,1),  and  therefore 

fnan,0 


Tn,0/an,0 


N( 0, 1) .  We  also  have  from  Lemma  1.3 
of  [9]  and  Theorem  1  that 

E{(T  0-E{Tn  J-Tn  n)2^ 
n,9  n,0  n,U 


|eue 


h.}  -  E{Ir  }  E{h  .}  |  <  4(a.+Bj 
j  J  Lj  J  11 


<  E{(g(N1+0nS1)-g(N1))2)li 


and  the  desired  result  follows  from  the  proof  of 
Lemma  lof[6],  Q.E.D. 

When  dependency  is  present  between  the  two 
processes,  we  can  obtain  an  analogous  result  to 
Lenina  1  if  the  noise  is  dependent  on  a  finite 
"window"  of  the  signal,  such  as  the  signal - 
dependent  noise  induced  through  reverberation 
effects.  The  extension  of  Lemma  1  to  this  signal- 
dependent  case  is  given  by  the  following: 


+2[(4+6E<(g(N1+enS1)-g(N1))2+6})  \  Y«/(2+fi)]‘4 
♦  lEfgdyo^j)}!  . 

where  £  y(V(2+6)  <  ^  and  .  1  =  1,2,...}  is  an 
i  =  l  1  1 

a-representation  for  the  6-acceptable  process 
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>  y  -f  ^  - 


(g(N )-g(N.. ) ;  i=l,2,...}.  Using  a  result 

from  [10],  we  conclude  from  assumptions  (c)-(c') 

and  (d)  that  E{(T  -E{T  J-T  n)2}  +  0,  which 
n  ,u  n,y  n,u 

yields  the  desired  result  from  [7,p.25].  Q.E.D. 
We  can  now  obtain  the  main  result: 

Theorem  3:  Suppose  that  the  hypothesis  of  Lemma  1 
or  Lemma  2  is  satisfied,  and  g  «  Then  g  is 
optimal  (in  the  sense  of  the  ARE)  if  and  only  if  g 
satisfies  (up  to  a  scale  factor) 

(A)  l  f  i(x.y)  +  fi(y,x)]g(y)dy  +  f'(x) 

j  =  l  J  J 

=  -f(x)g(x) 

if  and  {Sj}T=j  are  independent  and 

E{S^}  f  0,  or 

(B)  1  +  fi(y»x)]g(y)dy  +  f"(x) 

j=l  J  J  J 

=  -f(x)g(x) 


l  /[f,(x,y)+f.(y,x)]g(y)dy+  h(x)  =  -f(x)g(x), 
j=l  J  J 


then,  as  in  [4-6],  we  might  wish  to  approximate 
the  nonlinearity  g  by  nonlinearities  g^  which  are 

solutions  to  the  equations 


m 

j=l 


/[fJ*(x,y)+fj(y,x)]gm(y)dy  +  h(x)  =  -f(x)gm(x). 


In  this  case  we  would  then  hope  that  the 
convergence  in  some  appropriate  sense  of  the  gm 

would  lead  to  an  optimal  nonlinearity.  This 
question  is  answered  in  the  following: 

Theorem  4:  Under  the  hypothesis  of  Lemma  1  or 
Lemma  2,  if  there  exists  a  g  «#such  that  a 

subsequence  {gmk}£=1  of  (gm}~=1  satisfies 

gm  (N])-g(Ni) -^0  in  L?+6  ,  then  g  is  optimal, 
k 

Proof:  We  may  repeat  the  first  part  of  the  proof 
of  Proposition  3  of  [5],  and  conclude  that  it  is 
sufficient  to  show 


if  {N.;  i  =  l ,2 , . . . }  and  {S^ ;  1*1,2,...}  are 
independent  and  E{S ^ }  =  0,  where  f j ( • , • )  is  the 
joint  density  of  Nj  and  and  f  is  the 

univariate  density  of  or 

(0  J  J  [fj(x,y)  +  fj(y,x)]g(y)dy  +  Jy  — f(x,y)dy 

=  -f(x)g(x) , 

if  fj yg(x)  f(x,y)dxdy  f  0,  or 


(0)  l  /[f^x.y) +f .(y,x)]g(y)dy 
j=l  J  J 

2 

+  Jy2-%  f(x,y)dy  *  -f(x)g(x), 

3x 

if  jjyg(x)  f(x,y)dxdy  =  0,  where  f ( - , * )  is 
the  joint  density  of  Nj  and  Sj. 

Proof:  In  view  of  Theorem  2,  the  proof  of 
Theorem  3  of  [6]  may  be  repeated.  Q.E.D. 

Because  of  the  assumption  of  the  existence  of  a 
signal  density,  the  above  results  do  not  directly 
apply  for  the  case  of  a  constant  signal.  However 
a  similar  modification  of  the  methods  of  [4,5] 
may  be  employed  to  show  that  the  appropriate 
condition  for  the  nonlinearity  g  in  the  constant 
signal  case  is  given  by  equation  (A)  of  Theorem  3. 
Thus  as  in  [6]  we  note  that  if  the  signal  and 
noise  are  independent,  the  optimal  nonlinearity 
for  the  case  of  a  random  signal  with  nonzero  mean 
is  the  same  as  that  for  the  constant  signal  case. 

As  in  [4-6],  the  above  Integral  equations  are  of 
nonstandard  form.  If  we  note  that  each  of  the 
equations  is  of  form 


mk 

[  Jf[f  Ax, y)+f  Ay, x)J[g  (y)-g(y)]5g(x)dxdy 
j=l  J  J  mk 

*  l  f[[fAx,y)+f.(y,x)]g(y)Sg(x)dxdy  *  0 

j'V1  J  J 


as  k-K»,  where  5g  is  an  arbitrary  zero  mean 

2+6 1 

variation  satisfying  E{|6g(N^)|  A}  <  where 
6^  >  6 .  Application  of  Lemma  1.3  of  [9J  to  the 

second  summand  above  shows  that  it  can  be  upper 
bounded  by 


[4+2(C,+C2+^qc:)]  l  af (2+6)  -  0, 

12  12  j=iyi 


where  Cj  =  E{  |g(Nj)  |2+l5}  <  <°  and 


v,2  -  E{|6g(N1) 

application  together  with  the  Schwarz  inequality 
shows  that  the  first  summand  can  be  upper  bounded 

,1/U+e)  ?  y[<2+«i)U+e)] 


,2+6 


}  <  <*.  Moreover,  a  similar 


[4+2(D1+D2+rD1D2)]1 


i  V 

j=i  J 


•  E{[3g(N1)]2}e/2  E{[g  (y-gfNj)]2}^2 


for  any  e>0,  where 

2+6, 

Di =  E{igm  (Ni)-g(Ni}i  ><<o 
~  k  2+6, 

D2  =  E{|6g(N1)|  M  <  ». 


Choosing  e  small  enough  so  that 
61/[(2+61)(l+e)]  >  6/ (2+6) , 


we  obtain  the  desired  result. 


Q.E.D. 

Note  the  conditions  on  the  optimal  nonlinearity  g 
and  the  u-representation  are  exceedingly  mild. 

For  example,  these  results  hold  if 

^{[gtN^+eSj)]4}  <  ®  for  all  6  *  [0,9^, 

°°  1/3 

[  a-  <  00  *  and  the  subsequence  of  Theorem  4 
i=l 

converges  in  L^.  We  remark  finally  that  the 
nonlinearities  gm  are  obtainable  through  standard 

Hi lbert-Schmidt  techniques  as  solutions  of 
Fredholm  integral  equations  of  the  second  kind. 

IV.  CONCLUSION 

We  have  considered  the  design  of  the  optimal 
detector  for  signal  detection  in  corrupting  noise, 
where  both  the  signal  and  noise  may  be  chosen  from 
a  large  class  of  strong  mixing  processes  and  may 
be  dependent  on  each  other.  We  have  seen  that 
this  design  reduces  to  the  solution  of  an  integral 
equation  in  which  knowledge  of  only  the  second- 
order  statistics  of  the  random  processes  involved 
is  required.  In  particular,  if  the  signal  is 
independent  of  the  noise  and  has  nonzero  mean, 
the  optimal  detector  is  the  same  as  in  the 
constant  known  signal  case. 
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